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Grounding Water: Building Conceptual Understanding through 
Multimodal Assessment 

Kerry L. Schwartz , 1 Holly Thomas-Hilburn , 2 and Arin Haverland 2 

ABSTRACT 

The world's population is growing by about 80 million people a year, implying an estimated increased freshwater 
demand of about 64 billion cubic meters annually (World Water Assessment Programme, 2009, Water in a Changing 
World: United Nations World Water Development Report 3, Chap. 1, p. 3-21). Groundwater depletion, which reduces the 
amount of valuable water available for drinking and food production, has become a global crisis. Decision-makers at all 
levels desperately need to understand the unseen system beneath their feet and its connection to the earth's hydrologic 
cycle. Yet teaching groundwater concepts is extremely challenging; foundational misconceptions about groundwater's 
location, movement, and connection to the hydrologic cycle are common. Quality, multimodal instruction, and assessment 
of groundwater topics will help to clarify elementary students' misconceptions and assist them in constructing accurate 
mental models of the groundwater system. This study examines student responses to different forms of assessment, 
including drawing prompts, to determine the best way to ascertain what students really know about the groundwater sys¬ 
tem, a key component of the larger water cycle system. The assessment tools included dichotomous, multiple-choice, and 
drawing questions used to elucidate students' conceptualization and understanding of the groundwater system. Assess¬ 
ment results show that students who are able to answer objective questions about groundwater are not necessarily able to 
demonstrate their knowledge; calling into question their conceptual understanding of the system. © 2011 National Associa¬ 
tion of Geoscience Teachers. [DOI: 10.5408/1.3604827] 


INTRODUCTION 

An understanding of the hydrologic cycle is not just 
about defining evaporation, condensation, and precipita¬ 
tion, but also about articulating how water changes states 
and moves from one part of the system to another. Water 
is undeniably fundamental for survival and as most of the 
world's populations are increasingly reliant on indefinite 
water supplies, an understanding of the groundwater sys¬ 
tem within the hydrologic cycle is essential for wise water 
use and good water stewardship worldwide. This research 
examines conceptions and misconceptions about ground- 
water held by elementary school students in the arid 
Southwestern United States, in an effort to identify the best 
way to assess conceptual knowledge of the groundwater 
system and its relationship to the hydrologic cycle. 

To ascertain students' understanding of the ground- 
water system, researchers elected to evaluate the extent to 
which fourth graders retained the overall "big picture" 
concepts of the location, use, storage, and movement of 
groundwater. In accord with the 4th grade earth science 
learning benchmarks (Arizona Department of Education, 
2005), the Arizona Water Festival (AWF) learning experi¬ 
ence focused on students at this grade level. Students liv¬ 
ing in the arid Southwestern United States are reminded 
daily of the lack of visible water in their environment, beg¬ 
ging the question: "Where does the water they use come 
from?" While student comprehension of the entire cycle 
deserves further analysis, this study focused on the com¬ 
plex and unseen groundwater system due to the disquiet¬ 
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ing lack of understanding of it as an essential part of the 
hydrologic cycle. 

CONTEXT 

The Importance of Grounding Water 

The world's population is growing by about 80 million 
people annually, implying an annual estimated increased 
freshwater demand of about 64 billion cubic meters (World 
Water Assessment Programme, 2009). Over the past cen¬ 
tury, groundwater withdrawal has grown to exceed natu¬ 
ral renewable groundwater storage (or safe yield) in many 
areas of the globe (Narasimhan, 2010). Groundwater with¬ 
drawn for agricultural use in Bangladesh, China, India, 
Iran, Pakistan, and the United States accounts for well over 
80% of global groundwater use (Shah et ai, 2007). The 
deep-well turbine pump, first deployed for agriculture in 
California in 1907, made it possible to lift large quantities 
of water from depths of tens of meters in wells, limited 
only by the availability of adequate energy (Freeman, 
1968). 

According to the U.S. Geological Survey, a century- 
long observational record of groundwater depletion exists 
for the United States major aquifer systems (McGuire et al., 
2003; Narasimhan, 2009). In two unconfined systems, one 
of which includes the aquifer system in the study region, 
withdrawals have been accompanied by tens of meters of 
permanent water decline with no prospects of recovery in 
the foreseeable future (Narasimhan, 2009). Three confined 
aquifer systems —the Dakota, Atlantic Coastal Plains, and 
California's San Joaquin Valley aquifers — have experi¬ 
enced 80% nonrecoverable compaction due to ground- 
water withdrawals (Narasimhan, 2009). Groundwater 
development in India presents another example of over¬ 
draft. The introduction of turbine pumps during the 1960s 
to enhance agricultural production was successful in meet¬ 
ing its goals, but also led to severe declines in water levels 
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and alarming depletion of groundwater storage over many 
parts of India (Narasimhan, 2006). 

Groundwater depletion has become a global crisis. In 
many regions where population is rapidly growing, imme¬ 
diate action must be taken and policy makers will be 
expected to make educated decisions on the inter-related 
topics of water supply, water quality, and water policy. 
Along the same lines, students who are just beginning to 
form their conceptions and understandings of the natural 
world may be the most important stakeholders as they are 
the future educators, voters, leaders, policy makers, finan¬ 
ciers, consumers, and water users. 

Of great concern is that groundwater is not regularly 
covered in school curricula, in spite of the fact that 78% of 
community water systems in the United States have 
groundwater as their primary source (U.S. Environmental 
Protection Agency, 2010). Researchers, who have only 
recently begun to take notice of how students are taught 
and learn about groundwater, note that teachers them¬ 
selves are often ill-prepared to teach the topic (Dickerson 
and Callahan, 2006) and that the term groundwater never 
appears in the National Science Education Standards 
(Dickerson et al., 2007). In the state of Arizona, where this 
study took place, the term groundwater is listed as one of 
four examples provided in a performance objective that 
states: Identify the sources ofivater within an environment (Ar¬ 
izona Department of Education, 2005). Although ground- 
water is only briefly mentioned in the science standards, 
understanding groundwater as an integral part of the 
hydrologic cycle is important, especially in arid states like 
Arizona, where groundwater is 44% of the water supply 
(Arizona Department of Water Resources, 2006). 


Groundwater, water beneath the earths' surface which 
saturates the pores and fractures of sand, gravel, and rock 
formations (U. S. Geological Survey, 2009), is unseen, and 
therefore difficult to visualize or articulate. Deeply held, 
naive conceptions based on what can be seen at the land 
surface interfere with groundwater instruction. Such con¬ 
ceptions develop from formal instruction and emerge from 
errors or misleading representations in texts, lectures, and 
inappropriate or misapplied practical experiences through¬ 
out the student's history (Dickerson and Dawkins, 2004). 
Instructional tools that use concrete representations of con¬ 
cepts as abstract as groundwater must remain as complete 
and accurate as possible in order to serve a useful purpose 
(see Fig. 1). 

Oversimplified or carelessly prepared models or 
graphics may prompt students to develop disconnected, 
isolated notions of groundwater concepts that yield an 
incomplete and inaccurate mental image (Dickerson and 
Dawkins, 2004). 

Building a Foundation for Conceptual 
Knowledge Acquisition 

The Arizona Water Festival program aims to provide 
children with "basic water literacy," introducing fourth 
grade students to the topics of watersheds, groundwater, 
the water cycle, and water conservation through stand- 
ards-based, hands-on activities both in the classroom and 
at the Water Festival itself—a two-hour field experience. 
Through formative evaluation of the Arizona Water Festi¬ 
val program over the last decade, coordinators have 
improved both the instructional method incorporating 
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FIGURE 1: A diagram of the groundwater system shows groundwater between the grains of sand and gravel under the 
ground, connected to surface water, and connected to the human system through pumping (Howe and Schwartz, 2007). 
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varied learning styles and assessment tool. Neuro-cogni¬ 
tion research supports the need for multimodal methodolo¬ 
gies in education. "Intermodal redundancy theory" states 
that learning is facilitated when information about the 
exact same subject is presented via two or more modalities 
(Bahrick and Lickliter, 2000). The "modality effect" (Pen¬ 
ney, 1989) suggests that the processing mechanisms are 
specific to either the auditory or the visual stream, and 
each stream has different properties and capabilities, 
resulting in different ways of representing information. 

Dickerson et al., (2007) suggested that a students' 
understanding of groundwater can be improved by the use 
of hands-on learning materials that enhance a students' 
understanding of groundwater by expanding their spatial 
reasoning about the topic. Similarly, Hobson et al. (2010) 
found that using models and representations helped chil¬ 
dren to understand spatial concepts, such as placement, 
which they may otherwise have found difficult to conceive 
or illustrate. Reinfried (2006) also discovered that college 
students who were instructed using mental model-build¬ 
ing methods, including hands-on models, were more likely 
to improve their understanding of groundwater systems 
than students who learned about groundwater through 
lectures and curricula that excluded mental model-build¬ 
ing. The need to identify students' preconceptions about 
groundwater is critical to providing effective instruction. 

Teaching in a multimodal fashion augments learning 
potential but necessitates multimodal evaluation to accu¬ 
rately measure student learning. Multimodal assessment is 
a credible and widely accepted method for gauging learn¬ 
ing and aptitude (Prain and Waldrip, 2006). The benefits of 
using a multimodal evaluation are that knowledge and 
understanding can be assessed through multiple routes 
(O'Bryne, 2009). Waldrip et al. (2006, p. 87) defined 
"multimodal" as "the linked use in science discourse of dif¬ 
ferent modes to represent scientific reasoning and 
findings." Multimodal assessments enable students to dis¬ 
play their knowledge and understanding through forms of 
assessment that highlight their strengths and provide a ro¬ 
bust review of a learner's comprehension. 

By allowing students to interact with groundwater 
models during lessons, educators facilitate students' 
"mental model" building. In their work on conceptual 
learning, Magnusson et al. (1997) suggested that learners 
typically grasp knowledge in terms of concepts, emphasiz¬ 
ing the need for conceptual science teaching. During the 
assessment process, rather than focusing on a rote answer, 
students are then able to convey areas where understand¬ 
ing is lacking while simultaneously referring to and sharing 
their own mental images of concepts learned. This knowl¬ 
edge, stored in the learners' individual mental models, may 
not be captured in a standard assessment instrument. 

Considerable research has been done on open and 
closed, oral and written questions, yet drawings are often 
overlooked or underutilized as a tool in assessment even 
though they serve as an alternative form of articulation 
(Dove, 2006). In this study, responses to drawing prompts 
are used as a means of determining what students truly 
know about the groundwater system. Arming (1997) main¬ 
tains that drawing can be a means to clarify thinking and 
effectively represent ideas to others. Similarly, in then- 
study of mental model building Glenberg and Langston 
(1992) proposed that pictures assist in the construction of 


mental models and parlay into working memory. They 
support relationships that are implicit and assist learners in 
articulating more complexity. 

METHODS 

Using results from the Arizona Water Festival interven¬ 
tion, the objective of this research was to determine the best 
way to assess what 4th grade students understand about 
the groundwater system. The overall water education expe¬ 
rience was delivered in multiple modes on the day of the 
event and bolstered by similarly multimodal pre- and post¬ 
festival lessons taught in the classroom. Our assumption 
was that the use of multimodal teaching methods, like 
those used in the intervention, provides the best instruc¬ 
tional practice for teaching about the groundwater system. 

Setting and Intervention 

The Arizona Water Festival program in 2009 trained 
622 volunteers to deliver engaging water education to 6924 
fourth graders and their 313 teachers. An AWF held out¬ 
doors at a community park is a highly structured and 
organized event in which as many as 500 students rotate 
through four learning activities in the morning and another 
500 rotate in the afternoon. Students move through four 
stations with their whole class, ranging from 15-35 stu¬ 
dents, where 2-4 trained volunteer instructors facilitate 
activities. Learning activities are 30 min in duration and 
address four topics: groundwater, watersheds, water cycle, 
and water conservation. In the AWF setting, students are 
fully engaged in learning because they are in a new setting, 
learning from new instructors, and exposed to interactive 
demonstrations and models that they would not likely be 
exposed to in the classroom. Before and after the AWF, 
teachers conduct pre- and post-festival lessons covering 
the same four topics in the classroom. The AWF water edu¬ 
cation program has reached 33,337 students in 20 Arizona 
communities, using multimodal instruction over a decade. 

The 2009 AWF Summative Evaluation, using pre- and 
post-student questionnaires, demonstrated statistically sig¬ 
nificant results for student gains in overall water content 
knowledge. Student knowledge about water increased pre- 
to post-water festival and their enthusiasm for water 
conservation and learning about water increased (Thomas- 
Hilburn and Schwartz, 2009). Furthermore, students in the 
classes of teachers who participated in the professional de¬ 
velopment workshop showed greater gains than those 
who did not. According to post-festival survey results, 
100% of participating teachers agreed that water festivals 
teach water concepts more effectively than the teachers 
could accomplish in their classrooms and 98% agreed that 
their students were more likely to conserve water after fes¬ 
tival attendance. Teachers also rated their students' reac¬ 
tion to the festival as "excellent" and felt that the water 
festival should be repeated in their community. Thus the 
AWF program is an effective instructional model as eval¬ 
uated through formative and summative assessment. 

The use of trained volunteer presenters who are not 
classroom teachers creates a situation in which presenters 
can be specifically trained in teaching techniques appropri¬ 
ate for that lesson. In a 2 hour training session, volunteers 
are trained to facilitate the lesson by first observing master 
facilitators model the lesson as they would with 4th grade 
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students and then presenting the lesson back to the master 
facilitators. As facilitators of learning, the volunteers use 
kinesthetic, visual, and inquiry methods to assist students 
in developing conceptual understanding. The facilitators 
ask questions that lead students to discover the main 
points of the lesson. 

Though the AWF intervention covers the topics of 
watersheds, groundwater, the water cycle, and water con¬ 
servation, the focus of this study is on a students' under¬ 
standing of the groundwater system and the water cycle as 
it relates to groundwater. The groundwater lesson has 
three inquiry-based sections using three different models 
to cover five main points: 

• Groundwater is located underground between the 
grains of sand and gravel. 

• Groundwater moves underground. 

• Groundwater is connected to surface water. 

• Groundwater is a part of the water cycle. 

• Groundwater is an important source of water for 
human use. 

During the 30 min lesson on the groundwater system, 
the facilitator questions the students directly on what they 
observe —a process designed to build strong foundational 
knowledge. Lessons incorporate checks for understanding 
at multiple points that enable the instructor to dispel com¬ 
mon misconceptions students hold about groundwater. 

In section one, students pour water through open- 
ended tubes that contain different earth materials and 
observe the effects of grain size on the rate at which water 
flows through each material. In section two, they interact 
with a cross-sectional flowing model that enables them to 
see water flowing laterally, due to the effects of gravity 
through the same materials they experimented with in sec¬ 
tion one. In section three, students use small groundwater 
basins complete with simulated lakes to experiment with 
the connection between groundwater and surface water, 
and further explore how water fills the pore space between 
particles underground. In both sections two and three, stu¬ 
dents observe groundwater's connection to the larger 
water cycle. Finally, students pump water up to the surface 
from wells with a hand pump, observing the effects on the 
groundwater system and establishing the connection to the 
human water supply. 

In addition to the core groundwater lesson, the water 
cycle lesson includes groundwater as one of nine storage 
places where water can go in the earths system. Other places 
include oceans, rivers, lakes, soil, glaciers, plants, animals, 
and clouds. Each storage place is represented by a different 
colored bead and a unique cube that students roll to deter¬ 
mine their next destination. Students "become" water mole¬ 
cules moving through the water cycle and record their 
unique journey with different color beads. By examining 
each student's color-coded bead bracelet, representing their 
unique water cycle journey, students see that water entered 
and left the groundwater system, thus establishing ground¬ 
water's connection to the larger hydrologic cycle. 

The prefestival groundwater lesson, using physical and 
whole-body techniques, also has students experimenting 
with the movement of water through different earth materi¬ 
als due to gravity. Students design experiments to learn 
how water moves through different earth materials (sand, 


gravel, and clay-rich soil), and then use their bodies as water 
molecules and earth materials to model that movement 
again. Greene's "spacing-effect in memory" stipulates that 
retention of items to be memorized is improved by repeti¬ 
tion not grasped en mass at one moment (Greene 1989). 

The prefestival water cycle lesson has students build¬ 
ing a water cycle in a jar and observing conditions over 
time, thus reinforcing the understanding of processes by 
which water changes form and moves through the water 
cycle. The post-festival lesson for both topics re-establishes 
groundwater as a source of the human water supply, con¬ 
necting the natural water cycle to the human water cycle. 
Delivering multimodal content over a 2 week period 
ensures that content is reiterated in an effort to cement the 
subject matter and make the overall groundwater knowl¬ 
edge salient. 

The Research 

The groundwater system remains an unobservable 
mystery to most, and many individuals do not consider it 
as part of the hydrologic cycle; yet arguably, groundwater 
is one of the most important parts of the cycle in the South¬ 
west. With an overall intent to truly ensure that students 
are building conceptual knowledge toward understanding 
the groundwater system, researchers decided to take a 
deeper look at the student assessments on the subject that 
include multiple choice and dichotomous questions, and 
drawings. Through multimodal assessment, researchers 
hoped to better understand the students concepts of the 
groundwater system. This research will shed light on the 
best way to gauge a students understanding of the ground- 
water system and its relationship to the hydrologic cycle. 
Table I outlines the study. 

Assessment Instrument 

The pre- and post-assessments for the AWF were 
designed to include a variety of question types, and to 
assess a student's knowledge of all four topical areas cov¬ 
ered at the water festival. Excellence in Environmental Educa¬ 
tion: Guidelines for Learning (PreK-12) (North American 
Association for Environmental Education, 2004) was used 
to ascertain the level of skill and knowledge appropriate 
for 4th grade. Because the students taking the test were 
diverse in their reading, writing and language abilities, the 
nonwritten drawing portion of the test was of particular 
importance to enable students to demonstrate their knowl¬ 
edge, regardless of language capabilities (Medina-Jerez 
et al., 2007; Rice et al., 2004). 

As part of an overall evaluation of the AWF program, 
the student assessment was given in the classroom by reg¬ 
ular classroom teachers who had the option of attending a 
professional development workshop to learn how to teach 
these lessons. Of the students in the sample, 43% (n=71) 
were instructed by teachers who attended the workshop. 

As the assessment instrument was intended primarily 
for use in program evaluation, the emphasis in design was 
on usability for teachers and students as a learning tool 
rather than on psychometric properties (which were not all 
together ignored). The assessment tool was crafted based 
on the lessons' key ideas in each of four topics: ground- 
water, the water cycle, water conservation, and water¬ 
sheds. Initially, questions were developed to test each of 
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TABLE I: Grounding water: Building a conceptual understanding through multimodal assessment. 



the several learning objectives, and then given to students 
as open-ended sentence stems for completion. The open- 
ended responses from 120 students were compiled for each 
question and then grouped into categories expressing the 
same idea. Some questions were omitted because student 
responses indicated that the question was not testing the 
intended learning objective. Responses for the final multi¬ 
ple choice questions were created using actual student lan¬ 
guage and ideas, with some editing for clarity. The intent 
was to generate answers that reflected genuine common 
student misunderstandings. For the drawing prompts, var¬ 
iations in wording were tested on a sample consisting of 
several classes (120 students total) to find the wording that 
generated the best student drawings without prompting 
students to answer in a particular way, or "giving away" 
the answer. 

This study focused only on those responses to items 
related to groundwater. The set of groundwater items 
consisted of one multiple choice question and two 
dichotomous questions in the objective portion of the 
assessment. Responses to two drawing prompts focused 
on the groundwater system and its' connection to the 
hydrologic cycle were also utilized. Assessment items 
included: 

1. Most groundwater is found in: 

a. a huge lake under the ground. 

b. a river far below the soil we walk on. 

c. between particles of sand or gravel. 

d. solid bedrock underground. 

2. Groundwater moves underground. True False 


3. Surface ivater is not connected to groundivater. True False 

4. Draiv your idea of what groundivater looks like in the 
space below. Think about where groundwater is found, 
how it gets there, and what happens to it over time. 
Draw as many things as you can think of that relate to 
groundwater and label your drawing. 

5. Draw a picture of the ivater cycle in the space below. 
Include and identify all the places that water can go in 
the natural earth system. Include all processes that drive 
the water cycle. Use arrows to show the movement of 
water from one place to another. Label the ways the water 
changes form when it moves from one place to another 
(for example, condensation, freezing) 

It was important that the assessment be relatively sim¬ 
ple to administer and score, as the scale of the program did 
not allow for large amounts of time to be given to the 
assessment of individual students (e.g., interviewing and 
drawing discussion). Participants were given a pretest by 
their teachers before they started the water festival unit. 
Following the pretest, students participated in the prefesti¬ 
val lessons, the Water Festival itself, and finally the post¬ 
festival in-classroom lessons. Post-tests were given to each 
participant by their teacher upon completion of the water 
science unit. The pre- and post-tests were identical, yet the 
multiple weeks between the two minimized the test, retest 
effects. Upon completion, post-tests were deidentified. The 
objective sections were entered into a database and linked 
to the drawings which were scanned and loaded into 
ATLAS.ri, a qualitative analysis software for analysis and 
scoring. 
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TABLE II: Groundwater drawing rubric. 


5 

Drawing correctly displays groundwater between grains of sand and gravel (or mixed with dirt soil), and shows movement 
of groundwater from one place to another, percolation, and well pumping. 

4 

Drawing shows a less complete understanding of groundwater, displaying groundwater in the correct location, but static, 
or inaccurate depiction of movement. Is correctly located under the ground or between grains of sand, but not related to 

any other water features or systems. 

3 

Groundwater movement is shown in some capacity, whether via pumping, percolation, or flow from one place to another, 

but groundwater appears as a large lake under the ground. 

2 

Groundwater is correctly located underground, but appears as a large lake or river under the ground. Groundwater is not 

shown to be a part of a system. 

1 

Shows some understanding of the location or movement of groundwater, but is not clear in their presentation of the location 

and movement as it relates to the rest of the system. 

0 

Does not answer or the answer shows no understanding of where groundwater might be found. 


Study Population 

Nearly 7000 4th grade students (ages 9-10) partici¬ 
pated in the water festival in 2009, the year of this study, 
from the cities of Payson, Cottonwood, Yuma, Sierra Vista, 
Tucson, and Fountain Hills. All teachers whose classes par¬ 
ticipated in a festival were invited to submit their students' 
pre- and post-tests for use in the study. Ultimately, 1474 
complete student records (records with both a pre- and 
post-test) were submitted. To account for variability in 
classroom setting and educator styles, three students from 
each classroom that submitted tests for the study were 
randomly selected to be included in the sample for draw¬ 
ing scoring and analysis. In a few cases, selected records 
(n= 5) appeared to be incomplete, and these were later 
eliminated. This resulted in a sub-sample of 163 students, 
creating a manageable number of drawings for scoring 
purposes. 

The average ethnic breakdown across the six school dis¬ 
tricts that these students came from was 54% Caucasian, 
25% Hispanic, 4% African American, 2% Asian, 2% Ameri¬ 
can Indian, and 13% designated as other (National Reloca¬ 
tion for People on the Move, 2011). Across the six 
communities represented, the average number of students 
on free or reduced lunch was 55%. However, the districts 
represented in the sample were diverse, with the percentage 
of students on free or reduced lunch ranging from 15%-93% 
(National Alliance for Public Charter Schools, 2011). The 
percentages of English language learners were also wide- 
ranging, from less than 1%-41%. 

RESULTS AND ANALYSIS 

Researchers used assessment data collected from a suc¬ 
cessful instructional model to understand the best way to 
assess student understanding of the groundwater system 
and its relationship to the water cycle. The four criteria for 
4th grade conceptual understanding of the groundwater 
system include the following: 

Students will: 

• Recognize that groundwater is located underground 
between the grains of sand and gravel. 

• Explain groundwater's movement underground. 

• Demonstrate groundwater's connection to surface 
water. 

• Illustrate groundwater as part of the water cycle. 


Student assessment data for this research included 
groundwater questions and groundwater and water cycle 
drawing scores. The groundwater question scores account 
for the students' ability to answer three objective questions 
about groundwater correctly. Groundwater drawing 
scores reflect the students' ability to draw an accurate 
depiction of the groundwater system based on the rubric 
in Table II. Water cycle drawing scores reflect the students' 
inclusion of the groundwater system as a place where 
water can go in the water cycle. SPSS and Excel were used 
to analyze data in each category. 

To generate groundwater drawing scores, a drawing 
rubric was created to facilitate scoring of student drawings 
and identify common student thinking about groundwater 
and the water cycle. The rubric was written in a collabora¬ 
tive process between two individuals. The first step identi¬ 
fied ambiguous wording in the rubric, which was revised 
and tested again by two raters who were then able to 
achieve a high inter-rater reliability (Pearson's r=0.7). Since 
the raters disagreed on a relatively small number of cases, 
the reviewers then reviewed these cases together, rescoring 
after discussion of possible interpretations of erroneous 
features of the drawing. The groundwater drawing rubric 
assesses three of the four criteria identified for 4th grade 
conceptual understanding of the groundwater system, 
while the water cycle drawing provides insight to the 
fourth criterion. 

Atlas.ti, a qualitative analysis software tool, was used 
to maintain researcher notes about interesting drawings 
and to assign a score to each drawing based on the five- 
point rubric (0-5), with 0 reflecting low complexity (little 
or no understanding of groundwater) and 5 reflecting high 
complexity (high understanding of groundwater). In this 
case, complexity refers to the student's understanding of 
the location of groundwater, its connection to a system, 
and its behavior and function in the system. Drawings 
were scored by two separate researchers who then 
reviewed all instances of disagreement to choose the cor¬ 
rect score. Drawing scores were then correlated to each stu¬ 
dent's objective scores. Pre- and post-test scores were 
recorded in a database in Excel 2007 and SPSSvl6 software 
was used to conduct statistical analysis. 

This study did not focus on student growth pre- to 
post-assessment, as researchers were most interested in 
determining students' conceptual understanding of the 
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groundwater system. That said, it may be instructive for 
the reader to know that groundwater drawing scores did 
significantly increase with the intervention: The mean 
score increased by 0.396, with a standard deviation of 1.46. 
The improvement was significant, t (163) =3.478, p=0.001. 
Likewise, the groundwater question scores also increased, 
although the number of questions in the sample (three) 
made that increase less significant. Additionally, students 
whose teachers participated in the professional develop¬ 
ment workshop (n=71, M=2.32, SD=1.43) performed better 
on the post-test than those whose teachers did not attend 
the workshop (n=93, M=1.87, SD=1.46). This difference 
was significant, t (162) = 1.98, p=0.049, though not nearly 
as significant as the overall difference in pre- and post-test 
scores for all students. 

To evaluate the effectiveness of the various forms of 
assessment, student groundwater question scores were 
compared to their groundwater drawing scores. Of stu¬ 
dents who answered the multiple-choice groundwater sys¬ 
tem question correctly (reported groundwater as being 
found underground between particles of sand or gravel) 
(n=61, or 37%) only 18% («=11) reflected this in their draw¬ 
ings, while 82% (n= 30) did not demonstrate this knowl¬ 
edge in their drawings despite having answered the 
objective question correctly. Of the students whose draw¬ 
ings correctly illustrated groundwater (n= 25) as being 
located underground between particles of sand and gravel, 
44% (n=Tl) answered correctly when asked where ground- 
water was found within the multiple choice section by 
reporting that groundwater was "found between particles 
of sand or gravel." 

Such a large discrepancy between students' answers to 
the multiple choice question and the ability to appropri¬ 
ately draw groundwater was surprising. Thus, the research 
team then calculated the difficulty and discrimination 
index for this question to eliminate the possibility that the 
question was poorly written or too easy. Using the upper 
and lower 27% of the sample (Kelley, 1939), both indexes 
were found to be 0.30, or of moderate discrimination and 
difficulty. Applying Wiggins and McTighe's validity test, 
the drawing assessments used in this study have a high 
degree of validity (Wiggins and McTighe, 2005). 

There was a similar inconsistency in the dichotomous 
questions. One would anticipate that students who 
answered the dichotomous question, "Groundwater 
moves under the ground" to be "true" rather than "false," 
would also draw groundwater moving in some way. How¬ 
ever only 34% (n=48) of the 139 students who answered 


this question correctly also indicated in their drawings that 
groundwater moves. Of those students who actually drew 
groundwater movement in their depiction of the ground- 
water system, 94% (n=48 of 51) correctly answered the di¬ 
chotomous question. Similarly, it could be expected that if 
students indicated the statement, "surface water is NOT 
connected to groundwater" was "false," they would also 
be able to draw some connection between groundwater 
and surface water. However, the ratio of incorrect draw¬ 
ings to answers was similar to that of the other questions, 
with just 39% (n=45) of the 114 students who answered the 
dichotomous question correctly showing a groundwater- 
surface water connection in their groundwater drawing. 
Yet again, the majority of students (82%, n= 45) who drew 
groundwater connected to surface water also answered the 
dichotomous question correctly. Overall, the dichotomous 
questions were somewhat less difficult (with difficulty 
indices of 0.84 and 0.75, respectively), which may be 
expected because of the nature of true/false questions and 
student guessing. However, the two questions still had rel¬ 
atively robust discrimination indices. The discrimination 
index for the question regarding the movement of ground- 
water is 0.24, and for the question regarding surface and 
groundwater connection, 0.39. 

To further explore the students' understanding of 
groundwater, specifically its connection to surface water 
and the water cycle, the researchers returned to students' 
drawings of the water cycle. The drawing prompt did not 
specifically ask about groundwater or imply in any way 
that students should include it. Though the drawings were 
initially scored with a holistic rubric similar to that of the 
groundwater prompt (see Table III), the rubric did not spe¬ 
cifically require the inclusion of groundwater to obtain a 
good score (though including groundwater would mean 
that students would likely score higher on the rubric). How¬ 
ever, students could not score less than a 4 if they included 
groundwater in their drawing. All water cycle drawings 
with a score of 4 or 5 were re-examined for the inclusion of 
groundwater. A small but significant number of students 
(18%, n=23) did include groundwater in their conception of 
the water cycle without being asked to do so. These stu¬ 
dents, as we might expect, scored higher on the ground- 
water drawings as well (M=2.30 as opposed to M=2.03 for 
those who did not include groundwater in the water cycle 
drawings.). Seventy-four percent (n=17) of those who 
included groundwater in the water cycle drawing also cor¬ 
rectly identified that groundwater is connected to surface 
water in the dichotomous question, compared to 69% 


TABLE III: Water cycle question rubric. 


5 

Labels and correctly shows five processes of the water cycle (e.g., evaporation, precipitation, condensation, percolation, tran¬ 
spiration, runoff, freezing) and draws at least five places water can go in the earth system (of the nine: plants, animals, soil, 

groundwater, ocean, lake, glacier, river, clouds). 

4 

Correctly shows and identifies 3 processes of the water cycle, and draws at least 3-4 places that water can go in the earth 

system. 

3 

Confuses some process names, but the drawing shows a basic understanding of how water moves in the earth system. 

Shows two or fewer places water can go in the earth system. 

2 

Confuses how water moves in the earth system, or shows disconnected parts of a cycle. 

1 

Shows that water moves in the earth system, but no understanding of anything much further than rain falling on the earth. 

0 

Does not answer or the answer shows no understanding of how water moves in the earth system. 
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(n=97) of those who did not include groundwater in then- 
water cycle drawing. In contrast, only 15% of those students 
who identified that groundwater and surface water are con¬ 
nected in the dichotomous question also drew groundwater 
into their drawing of the water cycle. This discrepancy again 
points to a disconnect between what students appear to 
know in answering objective questions and what they are 
able to demonstrate in drawings. 

In analyzing what students understand through draw¬ 
ings, researchers found that a comparison of pre- and post¬ 
test data is instructive. Many more students were able to 
express conceptual knowledge through drawing post-inter¬ 
vention. Only 2 students drew groundwater between grains 
of sand and gravel in the pre-test, while 11 were able to in 
the post-test. Twenty-six students showed groundwater 
moving underground in the pre-test, as compared to 48 in 
the post-test. For groundwater's connection to surface 
water, 24 and 4 students showed groundwater connected to 
the surface in their depictions of the groundwater system 
and water cycle, respectively, as opposed to 45 and 17 stu¬ 
dents in the post-test. Since it is very difficult to draw some¬ 
thing that you do not know about, we can assume that 
these gains show true gains in conceptual understanding. 

The need for robust multimodal assessment that 
includes drawings to assess true student learning is 
emphasized in Fig. 2. Paired bars represent multimodal 
assessment results of a specific criterion. The top, or Bar A, 
of each pair represents the total number of students who 


illustrated the listed concept in a drawing, with the darker 
areas showing those who also answered the related objec¬ 
tive question correctly. The lower, or Bar B, represents stu¬ 
dents who answered the objective question with regard to 
a particular concept correctly, with the darker area being 
those students who also reflected that same concept in 
their drawing. 

Overall, students who answer the multiple choice 
questions correctly are not very likely to reflect that knowl¬ 
edge in a drawing, whereas students who display knowl¬ 
edge in a drawing are relatively more likely to also 
respond correctly to an objective question testing the same 
knowledge. In summary, the students that could draw 
each of the concepts could answer objective questions cor¬ 
rectly 44%-94% of the time, while students that could an¬ 
swer each of the objective questions could only draw 
concepts correctly 15%-44% of the time. This data leads to 
the conclusion that the incorporation of drawing in assess¬ 
ment is an important discrimination tool in assessing con¬ 
ceptual understanding of the groundwater system. 

DISCUSSION 

The importance of the groundwater system to humans 
cannot be overstated, yet groundwater in most earth sci¬ 
ence instruction is often not included as the essential part 
of the hydrologic cycle that it truly is. The AWF interven¬ 
tion included a robust approach to groundwater system 


Groundwater... 


moves underground 


is between grains of sand and gravel 


is connected to surface water (in groundwater 
drawing) 


is connected to surface water (in water cycle 
drawing) 
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FIGURE 2: (Color online) Bar A in each pair shows the total number of students who illustrated the listed criteria cor¬ 
rectly in a drawing. Bar B of each pair shows the total who correctly answered the related objective question. Of those 
totals, darkened portions of each bar show the number of students who answered both the objective and drawing 
questions correctly. The same proportion is shown as a percent to the right of the bar chart. 
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instruction. This study uses multiple forms of assessment 
to analyze fourth graders' conceptual understanding of the 
groundwater system. 

To date, there is limited information on how students 
think about groundwater, and even less information on 
quality instructional methods that help to correct student 
misconceptions (Dickerson and Callahan, 2006). Studies 
that do exist focus on students who are much older than 
the fourth graders in this study who nonetheless are 
expected to have a basic understanding of groundwater 
(e.g., Dickerson et al., 2005; Dickerson and Dawkins, 2004; 
Rienfried, 2006). 

Dickerson et al. (2007) suggest, among other things, 
that improving groundwater education will require more 
attention to students' spatial reasoning techniques and 
appropriate assessment of student understandings. The 
use of models as part of the instructional portfolio and 
drawings as part of the assessment process are essential 
components of this recipe. In an analogous vein, Arming 
(1997, p. 219) suggested that "children instinctively use 
drawing in the same exploratory way that designers use 
sketching to 'converse with themselves' when generating 
ideas." McWilliam et al. (2008, p. 226) argued that creative 
capacity building should be repositioned in science, noting 
that "creativity is not the antithesis of scientific rigor but 
the core business of scientific thinking" which builds both 
academic and social capacity. In this study, data collected 
from the multimodal instruments that include the use of 
drawing prompts, reflects the Dickerson, Arming, and 
McWilliam approaches. 

The analyses of multiple student drawings allow educa¬ 
tors to ascertain even more fully how students think about 
the groundwater system. Only seven students demonstrated 
what would be considered a full understanding, by includ¬ 
ing groundwater in their drawing of the water cycle and 
connected to the surface in their drawing of groundwater. 
Figure 3 shows the pair of drawings for one such student. 
The two drawings together help build a more complete pic¬ 
ture of how this student understands groundwater. For 
instance, the placement of groundwater in the water cycle 
drawing may have been misinterpreted as a surface feature 
without the confirmation of groundwater's location in the 
drawing of the groundwater system. While one could 
choose to believe that any student who can correctly answer 
multiple choice questions has learned the content, looking at 
many forms of evidence of their understanding may reveal 
larger gaps than previously imagined. 

Since drawing prompts have a much higher degree of 
validity than either type of objective question, it is likely 
that they represent a more accurate picture of students' 
understanding of the groundwater system. Also, data 
show that if students could draw the concept, they were 
more often than not able to answer the related objective 
questions correctly, while answering objective questions 
correctly did not necessarily lead to the ability to draw the 
concept correctly. This supports the idea that drawing 
analysis is a valid and useful instrument for assessing stu¬ 
dent understanding. 

Contradictions in Multimodal Assessment Results 

Of particular interest are those students whose draw¬ 
ings and answers to the multiple choice question do not 
express the same conclusion. For example, in this study. 


FIGURE 3: A Water cycle drawing (above) and a ground- 
water drawing (below) from one student provides a more 
complete picture of the students' understanding of the 
groundwater system and its relationship to surface water. 

87% of students who were able to either draw or answer a 
multiple choice question about the location of groundwater 
were not able to do both. Similarly, while 70% («=114) of 
students were able to correctly answer that groundwater is 
connected to surface water in a dichotomous question, less 
than half of those (54 students, 47%) were able to draw 
that connection in either the water cycle or the groundwater 
drawing. 

One possible reason for incorrect drawings coupled 
with correct responses to objective questions is that stu¬ 
dents knew the answer "between the grains of sand or 
gravel" was correct, but when asked to draw that concept 
they built upon an erroneous construct for the ground- 
water system. One common drawing that students created 
showed a river of water flowing between a layer of sand 
and a layer of gravel, as in Fig. 4. This does in fact portray 
groundwater "between the layers of sand and gravel," but 
not between the grains. In conversations with middle 
school students, Dickerson and Dawkins (2004) found that 
students could sometimes state ideas that seemed correct, 
but with further questioning would reveal that they were 
using correct terminology to describe incorrect thinking. 
Thus, some students may be adept at recalling the correct 
answer through memorization, but may not actually 
understand the concept. 

Another possible reason for incorrect drawings is that 
it is very difficult to draw the groundwater system. A stu¬ 
dents' pretest drawings often reflected surface water rather 
than groundwater, or water that was simply "on the 
ground." Many professional depictions of groundwater 
used as visuals for public education show aquifers appear¬ 
ing as blue areas underground. It is artistically difficult to 
show water between grains of sand and gravel, and, in 
reality, groundwater is clear and difficult to see in a 
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FIGURE 4: A students drawing depicting a layer of 
water between layers of sand and gravel. While this stu¬ 
dent may correctly identify groundwater as "between the 
grains of sand and gravel," the drawing reveals an incor¬ 
rect understanding of the groundwater system. 

drawing or model. Students who successfully depicted 
groundwater as between grains often used labels or inset 
magnified areas, as in Fig. 5, where they could show what 
water and sand grains mixed together might look like, 
much like the professionally produced drawing in Fig. 1. 
Students who had colored pencils or crayons at their dis¬ 
posal were better able to show where they thought 
groundwater would be located (all teachers were provided 
with colored pencils and were asked to allow students to 
use color, but the vast majority did not). 

Similar to the logistical challenge of drawing the 
groundwater system, it is likewise difficult to design a 
drawing prompt that will tell students what to draw with¬ 
out actually describing that system. In this study, research¬ 
ers tested several different prompts before deciding to 
request that students draw their own idea of what ground- 
water looks like. The final prompt mentioned several 
aspects that students should consider, such as where it is 
located, how it gets there, and what happens to it over time. 
To avoid giving students the answer, the research team did 
not explicitly ask them to show the movement or the con¬ 
nections between groundwater and other systems. But it 
raises the question: Would students think to depict movement in 
a static drawing as is expected in the drawing rubric? 

Though rare in the study's findings, some students 
were indeed able to draw the groundwater system cor¬ 
rectly and unable to answer the multiple choice question 


FIGURE 5: Drawing by a student who scored low on the 
objective portion of the assessment, but depicted ground- 
water accurately and with creative inclusion of prior 
knowledge. 


correctly. These students may have difficulty reading or 
comprehending multiple choice questions. English lan¬ 
guage learners, low proficiency readers, and nonlogical 
learners may be among those who display more success 
through a visual depiction of their understanding. In Fig. 
5, the students drawing reflects prior knowledge about 
what is underground and indicates an evolving under¬ 
standing of groundwater, even though this particular stu¬ 
dents' objective test results did not reflect this knowledge. 
Research by Xu et al. (2009) found that some students may 
choose to draw a more realistic view of a concept, while 
others may elect to draw a more abstract view; both views 
depict the learning experience. 

CONCLUSION 

This research focused on determining what 4th grade 
students comprehend about the groundwater system and 
its' relationship to the water cycle. Though instruction that 
follows best practice is of great importance, this study 
demonstrates that the ability to interpret student thinking 
and mental model building is paramount to ensuring the 
development of a strong foundation on which true under¬ 
standing of a complex earth system depends. Students' 
ability to conceptualize the groundwater system as evi¬ 
denced by drawing seems to be a much stronger predictor 
of content mastery than the ability to answer objective 
questions. 

Groundwater is the lifeblood of the arid western states, 
but groundwater alone cannot sustain the metropolises of 
the Southwest. In the region of this study, the Colorado 
River supplies water for cities and agriculture in seven U.S. 
western states and two states in Mexico. Without this 
water, cities such as Los Angeles, Las Vegas, Denver, Phoe¬ 
nix, Tucson, and Albuquerque will overdraft the aquifers 
underlying them to dangerous degrees. Yet all predictions 
point to reduced water flow in the Colorado River system. 
Woodhouse and Meko (2010) predict that for every 1.8°F 
(1°C) of warming in the future, the Colorado River flow is 
projected to decrease between 2% and 8%, according to 
paleoclimate records over the past two millennia. 

In the Southwest, characterized by an arid climate 
with highly variable precipitation patterns, surface water 
supplies are diminishing, and groundwater supplies have 
not recovered from past overdraft. Extended drought and 
climate change trends may significantly and permanently 
impact our water supply, and thus our region's ability to 
sustain the current quality of life and ecosystem health. It 
is critical that citizens are educated on the subject of our 
interconnected water resources, especially the unseen and 
least understood groundwater system, so that they can 
make wise decisions that will conserve resources to ensure 
the sustainability and economic vitality of communities in 
the Southwest. 

In teaching and assessing groundwater, students are 
often tested on whether or not they know the correct 
words for talking about the system. However, from this 
study, it is evident that the ability to choose the correct 
words to describe a system does not necessarily translate 
to a true understanding of the system. In light of the signif¬ 
icance of groundwater as a resource in the arid southwest, 
and the consequential decisions that will need to be made 
about groundwater systems in the next generation. 
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educating students to fully understand the system is essen¬ 
tial. The use of tools that truly assess conceptual knowl¬ 
edge is critical to obtain a robust understanding of how 
learners grasp and retain concepts. 

Implications 

Teachers not only need to include groundwater in their 
instruction of the water cycle, but also appropriately evalu¬ 
ate conceptual knowledge. The use of drawing to assess 
age-appropriate conceptual understanding of complex sys¬ 
tems like the groundwater system warrants further research. 
Drawings are underutilized as a tool in assessment even 
though they serve as an alternative form of articulation 
(Dove 2006). Drawings combined with student explanations 
could be a powerful tool to assess conceptual knowledge 
and identify misconceptions. Too often, quick and easy mul¬ 
tiple choice or dichotomous questions are used erroneously 
to attempt to assess conceptual knowledge. 

Of course the time that teachers currently have to 
gauge student understanding is limited, due to a strong 
focus nationally on reading, writing, and math skills, as 
well as the breadth of science content to be covered. The 
use of drawings requires the development of thoughtful 
rubrics and time to study what students' truly comprehend 
about a concept. One possible alternative to the completely 
student generated drawing is a picture-based multiple 
choice or short answer question. An example of this 
appears in Rienfried's 2006 study, in which students are 
asked to choose one of four block diagrams that best repre¬ 
sents their understanding of how groundwater deposits 
appear underground. This sort of question may provide a 
middle ground between ease of use and information 
provided. 

A variety of studies (Dickerson et al. 2005; Dickerson 
and Dawkins, 2004) have taken "snapshots" of students 
learning about groundwater at various age points, both 
before and after interventions that teach students about 
groundwater. However, if the aim is to educate the general 
public to be active, responsible, well-informed citizens able 
to make good water policy decisions, longitudinal studies 
of students who have received groundwater education at a 
variety of age-levels are necessary to avoid the common 
misconceptions that perpetuate. In other words, studies 
are needed to determine the point at which learners are 
able to synthesize their pre-existing knowledge into an 
accurate overall concept. A further study such as this that 
includes an opportunity for students to verbally explain 
their drawings could help build the instructional frame¬ 
work and assessment tools for teaching this complex, 
though important, subject. The need to clarify student 
ideas while using hands-on experimental model-building 
is clear in the Reinfried (2006) study. 

Finally, a re-emphasis on drawing in schools may 
improve students' abilities to express themselves through 
an art form and assist in assessing conceptual understand¬ 
ing of all content areas. Teachers will need professional de¬ 
velopment in this form of assessment. Just as artists 
themselves differ in representations of the same thing, 
some students may elect to draw a more realistic view, 
while others may elect to draw a more abstract view, yet 
both views depict the learning experience (Xu et al., 2009). 
The ability to draw can exhibit spatial intelligence and may 


provide great insight into conceptual understanding, but 
usually requires some instruction. The interpretation of 
student drawings will be enhanced by the inclusion of stu¬ 
dents' commentary. 
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